Mekodata 26 03 13 by krishna-yb · Pull Request #2 · yugabyte/langfuse

krishna-yb · 2026-03-13T09:11:04Z

What does this PR do?

Fixes # (issue)

Type of change

Bug fix (non-breaking change which fixes an issue)
Chore (refactoring code, technical debt, workflow improvements)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Refactor (does not change functionality, e.g. code style improvements, linting)
This change requires a documentation update

Mandatory Tasks

Make sure you have self-reviewed the code. A decent size PR without self-review might be rejected.

Checklist

I haven't read the contributing guide
My code doesn't follow the style guidelines of this project (pnpm run format)
I haven't commented my code, particularly in hard-to-understand areas
I haven't checked if my PR needs changes to the documentation
I haven't checked if my changes generate no new warnings (npm run lint)
I haven't added tests that prove my fix is effective or that my feature works
I haven't checked if new and existing unit tests pass locally with my changes

Apply the current tracing migration and UI updates on chore/tracing-only-ui, remove golang-migrate installation from the web image, and keep compose/docs aligned for cached local builds.

…r rebase

Route tracing repositories to TRACING_DATABASE_URL while keeping Prisma metadata migrations on the primary DATABASE_URL, and update local-build compose wiring to bootstrap tracing schema before worker startup.

Switch tracing repository imports to use a live tracing Prisma binding to avoid undefined $queryRaw crashes at runtime, and add targeted startup dependencies in local-build compose so web/worker wait for required infra services.

Rename the Yugabyte service, keep metadata/tracing init explicit, and standardize schema wiring for local startup stability.

Point metadata bootstrap at the tracked yb_postgres_data.sql file to avoid mounting a directory as /schema/data.sql and failing first-run initialization.

Restore and apply the previously stashed local compose adjustments and web page tweak on top of mekodata-26-03-13.

Made-with: Cursor

Bring back metadata/tracing schema init services in docker-compose.yml and keep local-build compose envs aligned with tracing-only Yugabyte setup.

kushidhar-in · 2026-03-16T05:21:13Z

docker-compose.yml

+      - "${WEB_HOST_PORT:-3000}:3000"
+    environment:
+      # Metadata DB (Prisma migrations + metadata tables)
+      DATABASE_URL: "postgresql://yugabyte:yugabyte@yugabyte-db:5433/langfuse"


hardcoded here, use this kinda syntax for allowing overriding from .env file
CLICKHOUSE_URL: ${CLICKHOUSE_URL:-http://clickhouse:8123}

kushidhar-in · 2026-03-16T05:21:28Z

docker-compose.yml

+    environment:
+      # Metadata DB (Prisma migrations + metadata tables)
+      DATABASE_URL: "postgresql://yugabyte:yugabyte@yugabyte-db:5433/langfuse"
+      TRACING_DATABASE_URL: "postgresql://yugabyte:yugabyte@yugabyte-db:5433/langfuse?schema=clickhouse"


same as here

kushidhar-in · 2026-03-16T05:22:13Z

docker-compose.yml

-    restart: always
-    depends_on: *langfuse-depends-on
+      # ClickHouse (kept for env compatibility; tracing repositories use Postgres)
+      CLICKHOUSE_MIGRATION_URL: "clickhouse://${CLICKHOUSE_CONTAINER_NAME:-langfuse-clickhouse}:9000"


need to remove the click-house urls & post-gress urls

kushidhar-in · 2026-03-16T05:23:38Z

docker-compose.yml

+      LANGFUSE_S3_MEDIA_UPLOAD_REGION: "${LANGFUSE_S3_MEDIA_UPLOAD_REGION:-us-east-1}"
+      LANGFUSE_S3_MEDIA_UPLOAD_ACCESS_KEY_ID: "${LANGFUSE_S3_MEDIA_UPLOAD_ACCESS_KEY_ID:-minio}"
+      LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY: "${LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY:-miniosecret}"
+      LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT: "http://minio:9000"


port is hardcoded, make it a variable & allow to fetch from .env
"http://localhost:${MINIO_API_PORT:-19090}"

kushidhar-in · 2026-03-16T05:23:48Z

docker-compose.yml

+      LANGFUSE_S3_BATCH_EXPORT_BUCKET: "${LANGFUSE_S3_BATCH_EXPORT_BUCKET:-langfuse}"
+      LANGFUSE_S3_BATCH_EXPORT_PREFIX: "${LANGFUSE_S3_BATCH_EXPORT_PREFIX:-exports/}"
+      LANGFUSE_S3_BATCH_EXPORT_REGION: "${LANGFUSE_S3_BATCH_EXPORT_REGION:-us-east-1}"
+      LANGFUSE_S3_BATCH_EXPORT_ENDPOINT: "http://minio:9000"


same as above

kushidhar-in · 2026-03-16T05:24:52Z

docker-compose.yml

-      - langfuse_clickhouse_data:/var/lib/clickhouse
-      - langfuse_clickhouse_logs:/var/log/clickhouse-server
+      # Metadata DB
+      DATABASE_URL: "postgresql://yugabyte:yugabyte@yugabyte-db:5433/langfuse"


urls are hardcoded

kushidhar-in · 2026-03-16T05:26:32Z

docker-compose.yml

+    - sh
+    - -c
+    - |
+      bin/yugabyted start \


@ashetkar could you please cross-review the configurations of yb image here please.

Drop clickhouse-only compose wiring and web startup hard-fail, and default legacy clickhouse env keys in web env schema so yugabyte-only local startup works.

Replace deprecated URL validation and provide defaults for ClickHouse worker env keys so local Yugabyte-only setups do not fail env parsing.

ashetkar · 2026-03-16T11:15:33Z

@krishna-yb As a standard practice, please update the description of the PR to include what was changed and why.

krishna-yb and others added 20 commits March 10, 2026 08:42

chore(tracing): postgres cutover updates and UI cleanup

7b83e25

Apply the current tracing migration and UI updates on chore/tracing-only-ui, remove golang-migrate installation from the web image, and keep compose/docs aligned for cached local builds.

chore(tracing): postgres cutover updates and UI cleanup

0522b62

Apply the current tracing migration and UI updates on chore/tracing-only-ui, remove golang-migrate installation from the web image, and keep compose/docs aligned for cached local builds.

fix(tracing): keep checkTraceExistsAndGetTimestamp postgres-only afte…

be1fe9f

…r rebase

rebase to master

300ac93

fix(tracing): split metadata and tracing Postgres connections

1ffe94f

Route tracing repositories to TRACING_DATABASE_URL while keeping Prisma metadata migrations on the primary DATABASE_URL, and update local-build compose wiring to bootstrap tracing schema before worker startup.

fix(tracing): stabilize tracing prisma runtime usage

5ede913

Switch tracing repository imports to use a live tracing Prisma binding to avoid undefined $queryRaw crashes at runtime, and add targeted startup dependencies in local-build compose so web/worker wait for required infra services.

migrate clickhouse to the yb

24ec5ed

merge pg_yb & clickhouse_yb

865b74b

clean the ybvoyager artifacts in the yb_postgres

f8cbd8a

clean up voyager metadata & statedata present in yb_clickhouse

91ff58e

chore: simplify yugabyte compose init flow

6ff7f48

Rename the Yugabyte service, keep metadata/tracing init explicit, and standardize schema wiring for local startup stability.

fix(compose): use yb_postgres_data.sql for metadata init

aa730cc

Point metadata bootstrap at the tracked yb_postgres_data.sql file to avoid mounting a directory as /schema/data.sql and failing first-run initialization.

Merge branch 'chore/tracing-only-ui' into mekodata-26-03-13

9acfae5

chore: apply stashed compose and web updates

2f099d7

Restore and apply the previously stashed local compose adjustments and web page tweak on top of mekodata-26-03-13.

Merge branch 'chore/yugabyte-compose-cleanup' into mekodata-26-03-13

ef6a901

Made-with: Cursor

fix(compose): restore schema init services and align local-build envs

18cde0f

Bring back metadata/tracing schema init services in docker-compose.yml and keep local-build compose envs aligned with tracing-only Yugabyte setup.

add basic e2e tests

67346b1

disabled flaky test cases

3fe009f

disabled flaky test cases

1618e9e

disabled flaky test cases

3ee6091

kushidhar-in reviewed Mar 16, 2026

View reviewed changes

kushidhar-in and others added 3 commits March 16, 2026 11:01

(test): add basic e2e tests

a9768ac

fixed hardcoded urls

fbddf60

(bug-fix): Fix hardcoded urls & replace with configurable variables.

b5f778d

krishna-yb and others added 3 commits March 16, 2026 08:06

fix(compose): remove clickhouse boot requirement for yb local

c48115d

Drop clickhouse-only compose wiring and web startup hard-fail, and default legacy clickhouse env keys in web env schema so yugabyte-only local startup works.

fix(worker): update clickhouse env schema defaults

577d9a4

Replace deprecated URL validation and provide defaults for ClickHouse worker env keys so local Yugabyte-only setups do not fail env parsing.

fix(sql): use PARTITION OF syntax for tracing child tables

ab1ff49

kushidhar-in added 2 commits March 16, 2026 17:36

fix(sql): use PARTITION OF syntax for tracing child tables

cbeb7f2

fix(compose): remove clickhouse boot requirement for yb local

20c7e48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mekodata 26 03 13#2

Mekodata 26 03 13#2
krishna-yb wants to merge 28 commits intomainfrom
mekodata-26-03-13

krishna-yb commented Mar 13, 2026

Uh oh!

kushidhar-in Mar 16, 2026

Uh oh!

kushidhar-in Mar 16, 2026

Uh oh!

kushidhar-in Mar 16, 2026

Uh oh!

kushidhar-in Mar 16, 2026 •

edited

Loading

Uh oh!

kushidhar-in Mar 16, 2026

Uh oh!

kushidhar-in Mar 16, 2026

Uh oh!

kushidhar-in Mar 16, 2026

Uh oh!

ashetkar commented Mar 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

krishna-yb commented Mar 13, 2026

What does this PR do?

Type of change

Mandatory Tasks

Checklist

Uh oh!

kushidhar-in Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

kushidhar-in Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

kushidhar-in Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

kushidhar-in Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kushidhar-in Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

kushidhar-in Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

kushidhar-in Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

ashetkar commented Mar 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kushidhar-in Mar 16, 2026 •

edited

Loading